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Preface 


Electronic  data  interchange  (EDI)  enables  the  Military  Traffic  Management 
Command  (MTMC)  to  exchange  information  with  a  host  of  dissimilar  informa¬ 
tion  processing  environments.  MTMC's  EDI  efforts  have  increased  significantly 
over  the  past  five  years  and  are  anticipated  to  grow  at  an  even  faster  pace  in  the 
future.  The  CONUS  Freight  Management  (CFM)  system.  Worldwide  liouse- 
hold  Goods  Information  System  for  Transportation  (WHIST),  and  Automated 
Carrier  Interface  (ACI)  system  are  exchanging  data  with  at  least  100  trading  part¬ 
ners,  with  hundreds  more  planned. 

Among  the  lessons  learned  to  date,  the  importance  of  maintaining  a  high 
level  of  data  quality  and  system  integrity  stands  out.  The  need  for  engineering 
better  data  quality  during  systems  development  and  ensuring  that  high  quality 
data  are  used  throughout  EDI  programs  have  been  well  known  throughout  the 
Department  of  Defense  (DoD).  Beyond  acknowledging  that  need,  MTMC's  past 
experiences  with  EDI  programs  indicate  the  subject  of  data  quality  requires  more 
attention  than  it  has  been  receiving. 

This  briefing  report  identifies  and  categorizes  recent  EDI  data  quality  prob¬ 
lems,  presents  industry's  approaches  to  resolving  data  problems,  and  recom¬ 
mends  measures  MTMC  should  take  to  improve  EDI  data  quality  as  it  continues 
to  develop  EDI  applications. 
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Agenda 


♦  Identify  and  categorize  recent  problems  with  the  quality 
of  EDI  data 

♦  Report  survey  results  of  DoD  and  industry  business 
practices  related  to  data  quality 

♦  Recommend  preventative  measures  and  reactive 
procedures  for  improvement 


In  this  briefing  report,  we  focus  on  recent 
data  quality  problems  and  their  causes.  We 
also  report  on  the  results  of  a  survey  of  elec- 
troruc  data  interchange  (EDI)  data  quality 
issues  within  various  Department  of  Defense 


(DoD)  organizations  and  then  industry  trad¬ 
ing  partners.  Based  on  the  results  of  that 
survey,  we  recommend  several  preventive 
measures  and  reactive  procedures  for 
improving  the  management  of  data  quality. 
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EDI  Data  Quality  Problems 


♦  Human  error 

►  Inadvertent  data  entries 

♦  Noncompliance 

►  Rules  and  procedures  known,  but  not  followed 

♦  Unregulated  requirement 

►  Procedures  do  not  exist 

►  Responsibilities  are  not  fixed 


The  Military  Traffic  Management  Com¬ 
mand's  (MTMC's)  data  quality  problems 
with  the  CONUS  Freight  Management  (CFM) 
system.  Worldwide  Household  Goods  Infor¬ 
mation  System  for  Transportation  (WHIST), 
and  Automated  Carrier  Interface  (ACI)  sys¬ 
tem  fall  into  five  categories.  Those  problems 
are  defined  on  this  and  the  next  chart. 

Human  error  problems  occur  when  a  rea¬ 
sonable  effort  has  been  made  to  reengineer  a 
process  and  develop  automated  system  tools 
that  minimize  the  potential  for  error,  but  they 
stiU  occur.  As  an  example,  making  a  data 
entry  error  while  t57ping  a  Government  BiU  of 
Lading  Office  Code  (GBLOC),  such  that  the 
newly  entered  GBLOC  incorrectly  matches 
one  in  an  edit  reference  table,  is  a  human 
error. 

Noncompliance  problems  occur  when  the 
user  or  analyst  building  an  interface  knows 


the  rules  and  procedures  but  elects  not  to 
comply  with  them.  An  example  of  this  error 
is  the  decision  by  some  shippers  not  to  sub¬ 
mit  an  Advance  Transportation  Control  and 
Movement  Docmnent  (ATCMD)  to  MTMC  as 
prescribed  by  DoD  4500.32-R,  Military  Stan¬ 
dard  Transportation  and  Movement  Procedures. 

Unregulated  requirement  problems  occur 
when  systems  analysts  do  not  research  the 
business  process  thoroughly  enough  to  deter¬ 
mine  if  a  requirement  is  supported  by  a  pub¬ 
lished  regulation  and  if  the  responsibility  for 
satisf5dng  the  requirement  are  imambiguous 
in  the  regulation.  An  example  of  this  type  of 
problem  is  the  recent  realization  that  installa¬ 
tion  transportation  offices  do  not  intend  to 
enter  accessorial  charges  on  personal  prop¬ 
erty  government  biUs  of  lading  (GBLs)  for 
shipments  they  receive  because  they  had 
never  been  required  to  do  so  and  were  not 
staffed  to  perform  the  work. 
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EDI  Data  Quality  Problems 
(continued) 


♦  Inadequate  LCM 

►  Nonexistent  programs 

►  Weak  or  ineffective  programs 

►  Programs  not  enforced 

►  LCM  practices  not  applied  to  smaller  programs 

♦  Inadequate  open  systems  communication 

►  Interface  coordination  not  formal/ineffective 

►  Global  standards  nonexistent/not  applied 

►  Global  systems  analysis  not  applied 


The  most  predominant  cause  of  data 
quality  problems  stems  from  a  lack  of  disci¬ 
pline  in  MTMC's  life-cycle  management  (LCM) 
of  automated  systems.  Furthermore,  LCM 
practices  are  not  required  when  project 
investment  is  small.  LCM  practices  should  be 
required  whenever  EDI  is  used  regardless  of 
the  size  of  the  project.  An  example  of  this 
type  of  problem  is  the  absence  of  documented 
operational  test  and  evaluation  requirements. 
In  another  example,  data  elements  needed  for 
an  interface  with  WHIST  were  optional  for 
the  data  base  providing  the  information  but 
were  mandatory  for  the  system  receiving  the 
information. 

The  last  category  of  problems  is  common 
among  organizations  that  move  from 


stovepipe  systems  to  systems  that  embody 
data  sharing  and  open  systems  architectures. 
These  problems  are  usually  exposed  when  an 
organization  moves  into  an  EDI  environment. 
It  is  a  problem  we  have  labeled  as  "inadequate 
open  systems  communication."  An  example  of 
this  t5q)e  of  problem  is  the  absence  of  well- 
coordinated  standard  reference  files  prior  to 
implementing  a  data  exchange,  such  as 
between  the  CFM  system  and  DoD  shipper 
systems  supplying  shipment  data.  Accred¬ 
ited  Standards  Committee  X12  EDI  standards 
do  not  dictate  standard  reference  files  and 
application  edits,  so  data  elements  such  as 
city  names,  commodity  codes,  and  standard 
point  location  codes  may  appear  in  many  dif¬ 
ferent  formats. 
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Organizations  Surveyed 


♦  DoD  Information  Resource  Management  College 

♦  DoD  Center  for  Data  Quality 

♦  PRC  Data  Quality  Engineering  Department 

♦  Peat  Marwick  Data  Warehousing  Engineering  Department 

♦  Institute  of  Internal  Auditors  Research  Foundation 

♦  3M  Company 

♦  Proctor  &  Gamble  Company 


In  an  attempt  to  imderstand  how  other 
organizations  have  resolved  their  data  quality 
problems,  we  surveyed  several  different 
organizations.  We  used  personal  interviews, 
telephone  calls,  and  reports  and  other  litera¬ 
ture  from  those  organizations  to  obtain  that 
imderstanding.  Some  of  these  organizations 
are  heavily  involved  in  EDI  and  data  quality 
engineering,  some  have  encoimtered  many  of 


the  same  problems  that  face  MTMC  today, 
while  others,  such  as  the  Defense  Information 
Systems  Agency  (DISA),  are  working  on 
global  standards  for  DoD.  We  foimd  no  "sil¬ 
ver  bullet  solutions,"  but  did  vmcover  a  few 
ideas  that  may  help  MTMC  to  improve  its 
data  quality. 
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Survey  Results 


♦  Data  quality  engineering  tools 

►  U.S.  Marine  Corps 

►  PRC  Data  Quality  Engineering  (DQE) 

►  QDB™  Solutions 

►  DB  STAR 

►  WIZSOFT 

♦  Industry  lessons  learned 

►  Paper  conversions  to  EDI 

-  Will  be  prone  to  error 

-  Requires  business  process  reengineering 

►  EDI  transaction  monitoring 

—  Requires  full-time  human  resources 

-  Audit  analytical  tools 


In  the  technology  arena,  a  number  of 
promising  data  quality  engineering  tools  are 
available  in  the  marketplace.  DoD,  and  pre¬ 
sumably  MTMC,  has  access  to  some  tools 
without  having  to  buy  them.  Marine  Corps 
recently  had  PRC  and  SRA  build  a  "Data 
Quality  Engineering"  tool  that  is  used  to  ana¬ 
lyze  data  bases  of  interfacing  systems  and 
indicate  potential  data  quality  problems.  This 
tool  requires  some  progranruning  experience 
to  operate  and  a  little  practice  before  it  can  be 
used  effectively.  A  more  robust  capability 
can  be  obtained  on  a  trail  basis  from  the  DoD 
Center  for  Data  Quality.  It  includes  a  sophis¬ 
ticated  tool  (QDB  Solutions)  and  trained 
analysts  that  will  conduct  the  analysis  and 
assist  with  tihe  corrective  actions.  In  addition 


to  QDB  solutions,  other  tools  such  as 
DB  STAR  and  WIZSOFT  offer  similar 
capabilities  at  competitive  prices.^  These 
auditing  tools  can  also  be  used  to  check  data 
bases  for  quality  problems  before  and  after 
implementing  a  new  interface. 

Discussions  with  commercial  enterprises 
that  are  using  EDI,  such  as  3M  and  Proctor  & 
Gamble,  revealed  one  common  theme:  the 
conversions  of  paper  to  EDI  requires  a  signifi¬ 
cant  effort  to  ensxire  good  data  quality  and 
will  almost  always  require  a  business  process 
redesign  before  implementation.  Addition¬ 
ally,  EDI  requires  substantial  resources  to 
monitor  daily  data  exchanges  and  to  resolve 
problems  with  trading  partners. 


^  Joe  Celco  and  Jackie  McDonald,  "Don't  Warehouse  Dirty  Data,"  Datamation,  15  October  1995. 
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Preventive  Measures  Guide 


♦  Enforce  sound  LCM  practices 

♦  Develop  formal  interface  controls 

♦  Implement  configuration  management 

♦  Identify  regulatory  changes  needed  during  concept 
development 

♦  Include  internal  controls  and  utilities  to  fix  corrupted  data  in 
system  design  requirements 

♦  Develop  standard  reference  files  and  standard  edits  among  all 


trading  partners 

♦  Establish  standing  Executive  Steering  Groups 

♦  Develop  centralized  data  quality  management  review  process 


The  most  important  preventive  measure 
is  to  follow  sound  LCM  practices.  Those 
practices,  however,  should  be  approached 
from  a  global  perspective.  Most  information 
management  organizations  realize  LCM  pro¬ 
cedures  are  not  glamorous,  easy,  inexpensive, 
or  quick  to  perform,  but  almost  all  realize  that 
problems  are  postponed  if  corners  are  cut  in 
this  area.  Two  aspects  of  LCM  require  special 
attention.  Business  processes  should  be 
redesigned  prior  to  EDI  development  and 
data  quality  design  requirements  should  be 
met  as  an  integral  part  of  the  design  phase. 

The  second  preventive  measure  is  to  for¬ 
malize  aU  interfaces  with  memorandums  of 
understanding  (MOUs),  interservice  support 
agreements  (ISAs),  or  interface  control  docu¬ 
ments  (ICDs).  Do  not  assume  anything  when 


working  through  the  analysis  of  a  planned 
interface.  Document  every  agreement  and 
qualified  stipulation  made  during  the  proc¬ 
ess.  When  developing  and  implementing  the 
interface,  document  all  steps  that  have  been 
taken  to  ensure  data  quality. 

Implement  formal  configuration  man¬ 
agement  for  software  and  hardware  configu¬ 
rations  and  for  the  entire  systems 
development  and  change  process.  The  con¬ 
figuration  of  formal  documentation  should 
include  ICDs  that  must  be  in  place  before  the 
software  is  designed  and  coded.  To  enhance 
trading  partner  communications,  configura¬ 
tion  management  boards  (CMBs)  should  have 
members  from  all  trading  partners;  MTMC 
should  also  seek  membership  on  boards  of 
systems  that  it  interfaces  with. 
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Preventive  Measures  Guide 

♦  Enforce  sound  LCM  practices 

♦  Develop  formal  interface  controls 

♦  Implement  configuration  management 

♦  Identify  regulatory  changes  needed  during  concept 
development 

♦  Include  internal  controls  and  utilities  to  fix  corrupted  data  in 
system  design  requirements 

♦  Develop  standard  reference  files  and  standard  edits  among  all 
trading  partners 

♦  Establish  standing  Executive  Steering  Groups 

♦  Develop  centralized  data  quality  management  review  process 

V  ^ 

Identify  regulatory  changes  up  front. 
Given  the  lead-time  needed  for  most  regula¬ 
tory  changes,  they  should  be  identified  early 
in  the  development  process  so  they  are  ready 
^vhen  the  system  is  implemented.  It  is  a  good 
practice  to  simultaneously  incorporate 
interim  regulatory  agreements  into  MOUs  or 
ISAs  in  the  event  the  regulatory  process  falls 
behind  system  development. 

Each  internal  control  should  be  listed  as 
part  of  the  requirements  specifications,  just  as 
functional  and  technical  specifications  are 
included.  Unfortunately,  mtemal  controls  are 
often  an  afterthought,  not  a  development 
requirement.  Numerous  incidences  suggest 
that  poor  internal  control  programs  often  lead 
to  fraud  and  information  seciuity  problems. 
Additionally,  even  though  most  system 
developers  strive  to  minimize  data  quality 
errors,  data  errors  will  periodically  contami¬ 
nate  a  data  base.  To  offset  those  errors,  utili¬ 
ties  should  be  included  in  the  system 


configuration  for  correcting  data  bases,  recon¬ 
stituting  data  operations,  and  recovering 
data. 

Develop  standard  reference  files  and 
edits  tihat  aU  trading  partners  can  apply. 
Without  those  files  and  edits,  data  quality 
problems  are  guaranteed  to  occur  and  even¬ 
tually  contaminate  a  data  base.  Even  if  there 
are  only  two  trading  partners,  insist  on  using 
global,  or  at  least  DoD-wide,  standard  refer¬ 
ence  files.  Data  sharing  often  starts  with  two 
parties  and  expands.  Trading  partner 
agreements  should  include  a  commitment  by 
each  party  to  edit  both  outboimd  and 
inbovmd  transactions. 

During  system  design  or  development  of 
interfaces,  establish  a  problem  resolution 
body  for  problems  that  the  cor\figiuration  con¬ 
trol  board  or  affected  parties  cannot  resolve. 
The  use  of  Executive  Steering  Groups  (ESGs) 
will  often  reduce  or  eliminate  crisis  and  spe¬ 
cial  interest  communications,  as  well  as  pre¬ 
clude  the  "quick  fix"  plans  that  could  have 
been  avoided  by  an  established  procedure  for 
elevating  problems  to  a  level  where  they  can 
best  be  resolved. 

These  preventative  measures  require 
centralized  management  oversight  to  be 
effective.  A  MTMC  quality  review  program 
should  be  established  for  each  major  EDI  pro¬ 
gram  to  ensure  that  preventative  measures 
for  data  quality  problems  are  considered  dur¬ 
ing  the  development  process. 
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Reactive  Measures  Guide 


♦  Identify  problem 

♦  Organize  problem  resolution  team 

♦  Implement  interim  procedures  and  data  correction,  if  required 

♦  Use  configuration  management  procedures  to  implement  solution 

♦  Initiate  MOUs,  ISAs,  ICDs,  or  regulatory  changes  as  needed 

♦  Reexamine  LCM  process 

♦  If  not  resolved,  refer  to  CMB,  TSRC,  or  ESG 


Reactive  measures  are  the  procedures 
that  should  be  followed  to  resolve  data  qual¬ 
ity  problems  if  preventive  measures  fail.  The 
first  step  is  to  isolate  and  identify  the  prob¬ 
lems. 

In  most  cases,  a  team  will  be  needed  to 
develop  the  solutions.  The  team  should 
include  personnel  with  the  expertise,  both 
functional  and  technical,  to  analyze  the  prob¬ 
lems,  as  well  as  to  understand  their  impact  on 
the  external  trading  partners.  AU  potentially 
affected  trading  partners  should  be  notified 
of  the  data  problems.  The  team  may  also  find 
a  need  to  implement  interim  procedures  and 
oversee  data  correction  efforts  while  more 
long-term  solutions  are  being  developed. 

Hardware,  software,  and  procedural 
changes  should  be  managed  in  accordance 
with  formal  configuration  management  pro¬ 
cedures.  Without  those  procedures,  the  risk 
of  implementing  imdocumented  software. 


hardware  configurations,  and  users  manual 
changes  is  greatly  increased. 

In  many  cases,  new  MOUs,  ISAs,  or  ICDs 
will  need  to  be  created  to  correct  the  prob¬ 
lems.  When  regulatory  changes  must  be 
made,  these  types  of  agreements  could  be 
used  to  implement  interim  measures  before 
the  actual  changes  are  published. 

A  reexamination  of  current  LCM  prac¬ 
tices  should  be  made  to  determine  if  the  data 
quality  problems  could  have  been  prevented 
by  better  LCM  practices  and  controls. 

When  the  team  cannot  agree  on  the  cor¬ 
rective  action  or  schedule  for  resolving  the 
problems,  problem  resolution  efforts  must  be 
elevated.  They  should  first  be  referred  to  file 
CMB  and  then,  if  necessary,  to  MTMC's 
Transportation  Systems  Review  Covmcil 
(TSRC).  If  the  problems  still  are  not  resolved, 
they  should  be  referred  to  tiie  ESG  as  a  matter 
of  final  recourse. 
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As  in  the  case  of  preventative  measures, 
resolving  data  quality  problems  also  requires 
central  management  oversight.  Since  EDI 
often  crosses  organizational  boimdaries,  reso¬ 
lution  requires  more  resources  than  available 
through  a  single  program  management  office. 
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Recommendations 


♦  Strengthen  LCM  practices 

♦  Emphasize  business  process  redesign 

♦  Implement  global  coordination  practices 

♦  Develop  standard  reference  files  and  edits 

♦  Explore  and  test  technical  data  quality  management  tools 
and  professional  services 

♦  Employ  sufficient  human  resources  for  EDI  data  quality 
monitoring 

♦  Obtain  leadership  commitment  and  manage  data  quality 
centrally 


In  summary,  MTMC  can  improve  EDI 
data  quality  by  implementing  the  key  recom¬ 
mendations  shown  on  this  chart.  MTMC 
should  review  current  LCM  practices  and 
strengthen  those  practices  that  contribute  to 
better  quality  control  assurance.  Prudent 
LCM  practices  should  be  used  for  all  EDI 
development  projects  regardless  of  the  estab¬ 
lished  regulatory  investment  costs.  When 
converting  a  data  exchange  to  EDI,  empha¬ 
size  business  process  redesign  instead  of 
simply  automating  current  practices.  Thor¬ 
oughly  coordinate  new  interfaces  and 
changes  to  MTMC's  automated  systems 
among  aU  trading  partners.  This  coordina¬ 
tion  should  include  development  and  imple¬ 
mentation  of  standard  reference  files  and 
edits.  Test  the  utility  provided  by  automated 
data  quality  tools  currently  available  to 
MTMC  and  its  trading  partners  for  little  or  no 
cost.  Employ  sufficient  human  resources  to 


monitor  EDI  trading  partner  transaction 
exchanges,  edit  rejects,  and  audit  data,  espe¬ 
cially  during  the  start-up  phase  of  an  EDI 
project.  Finally,  managing  information  assets 
requires  policy  development  and  compliance 
monitoring.  Many  organizations  accomplish 
this  task  by  establishing  a  standards  and 
quality  assurance  (QA)  staff  element.  In  most 
cases,  MTMC  has  delegated  QA  responsibili¬ 
ties  to  individual  program  management 
offices.  As  such,  there  is  no  single  organiza¬ 
tion  charged  with  the  overall  responsibility 
for  QA  and  related  policy  guidance.  Since 
data  quality  is  integral  to  the  product  MTMC 
delivers  to  its  customers,  it  is  recommended 
that  MTMC  formally  establish  a  structure  to 
establish  guidance,  monitor  compliance,  and 
direct  corrective  action  on  data  quality  mat¬ 
ters  related  to  EDI. 
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